Skip to content

Account Pool: model-family-aware routing and guarded account_uuid rewrite - #3053

Merged
SawyerHood merged 1 commit into
mainfrom
bb/account-pool-model-buckets
Sep 4, 2026
Merged

Account Pool: model-family-aware routing and guarded account_uuid rewrite#3053
SawyerHood merged 1 commit into
mainfrom
bb/account-pool-model-buckets

Conversation

@SawyerHood

Copy link
Copy Markdown
Collaborator

Standalone follow-up to the Account Pool stack (#3035, #3036, #3042, #3049); based on main.

Human comments

What was wrong

Account Pool selected accounts using only shared 5-hour and 7-day quota observations, even though Anthropic applies separate weekly limits to model families such as Fable. Scoped 7d_<bucket> headers were recorded as opaque diagnostics and never governed selection, the OAuth usage endpoint was not polled, and Claude Code request metadata could retain an account UUID belonging to a different OAuth account after routing. Part of #1552.

What changed

  • Replaced opaque bucket exhaustion diagnostics with fixed Fable, Sonnet, Opus, Haiku, and other weekly quota slots carrying utilization, reset, status, observation time, and source.
  • Added five-minute idle OAuth usage refreshes, immediate refresh after add or enable, scoped usage-limit parsing, and lazy account UUID backfill through the OAuth profile endpoint. Refresh failures preserve the last known state.
  • Parsed the request model once and made eligibility and reset tie-breaking use its governing family bucket. Scoped 429 responses now rotate accounts without globally exhausting the rejected account.
  • Stored account UUIDs during Claude Code import and OAuth sign-in, and guarded request rewriting to the selected account for recognized stringified-JSON and user_*_account_<uuid>_session_* metadata forms. Unrecognized, absent, non-JSON, or UUID-less requests remain byte-identical.
  • Added dynamic family columns to bb pool account list and updated the CLI skill, plugin guide, and configuration docs.
  • Added focused migration, usage parsing/refresh, family routing, scoped-header/429, UUID import/sign-in/backfill/rewrite, passthrough, CLI, and app regression coverage. There is no Plugin SDK or server/host-daemon wire change, so neither protocol version changed.

How you verified

  • pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool --continue --force — 6 tasks passed; 4 test files and 35 tests passed. The new routing, scoped-429, refresh, migration, and UUID tests fail against the prior implementation and pass with this change.
  • pnpm exec turbo run typecheck test --filter=@bb/templates --continue --force — 7 tasks passed; 7 test files and 43 tests passed.
  • pnpm exec turbo run typecheck --filter=@bb/server --continue — 5 tasks passed.
  • pnpm --silent bb plugin build plugins/account-pool — server and app bundles built successfully.
  • git diff --check origin/main...HEAD — passed after rebasing onto current origin/main.
  • Isolated dev-app proof with two OAuth accounts and switchThreshold=0.70: Account A had shared 7d at 43% but Fable at 80%; Account B had Fable at 7%. A real claude-fable-5-1 turn completed through Account B, then a real claude-sonnet-5 turn completed through Account A, proving family-only exhaustion does not disable the account globally. A redacted shape probe on a further real turn found metadata.user_id present without a recognized account UUID component, so the production path correctly preserved it byte-for-byte. The probe was removed, both imported test accounts were removed from the isolated store, and the dev app was stopped. Evidence is attached to BB-94.

Part of #1552

AGENT GENERATED

@SawyerHood
SawyerHood merged commit c2d5bd3 into main Sep 4, 2026
16 checks passed
@SawyerHood
SawyerHood deleted the bb/account-pool-model-buckets branch September 4, 2026 14:50
SawyerHood added a commit that referenced this pull request Sep 4, 2026
…3054)

Stack layer 2 (Codex pool track). Prerequisite: #3053 below it.
Core-only: adds plugin WebSocket routes so a provider pool can terminate
Codex's WebSocket-first transport at the hub.

## Human comments

## What was wrong

Plugin HTTP routes had no WebSocket upgrade surface. Codex derives its
WebSocket URL from the configured HTTP base URL and tries the Responses
WebSocket protocol first, so an Account Pool proxy mounted in a plugin's
existing `/http/` namespace could serve HTTP but could not accept the
corresponding upgrade. Codex therefore had to wait through failed
WebSocket attempts before falling back to HTTP SSE.

## What changed

- Added `bb.http.experimental_websocket(path, handler, opts)` and its
experimental socket, context, handler, and lifecycle types.
- Mounted exact-path upgrades in the existing
`/api/v1/plugins/<id>/http/` namespace while preserving ordinary HTTP
handling on the same path.
- Applied the existing `local`, `token`, and `none` auth modes to
upgrade requests, including header and query-token authentication.
- Isolated connection and event callbacks through the plugin invocation
boundary, serialized async callbacks per socket, normalized binary
frames to `Uint8Array`, and closed sockets with code 1012 when their
plugin generation reloads or is disabled.
- Added fake-host registration and socket-driving support, public
surface/audit entries, Plugin Guide references, and bumped the Plugin
SDK to 0.4.46.

There is no host-daemon wire contract change, so
`HOST_DAEMON_PROTOCOL_VERSION` does not need a bump. The existing
machine-auth proxy already forwards authenticated WebSocket upgrades and
retains its focused regression coverage.

## How you verified

The new server and fake-host regressions fail on `origin/main` because
the WebSocket API, registry, dispatch, and test driver do not exist
there. They pass on this branch and cover exact-path upgrade dispatch,
plain-GET behavior, HTTP/WebSocket coexistence, all auth modes, text and
binary frames, callback failure isolation, and code-1012 reload
shutdown.

- `pnpm exec turbo run typecheck test --filter=@bb/server
--filter=@get-bb/plugin-sdk --filter=@bb/plugin-api-map --continue` —
13/13 tasks passed; SDK 232 tests, API map 72 tests, server 2,209 tests;
all typechecks passed.
- `pnpm exec turbo run test typecheck --filter=@get-bb/plugin-sdk
--filter=@bb/plugin-api-map --filter=@bb/app
--filter=bb-plugin-plugin-api-docs --continue` — 11/11 tasks passed; app
3,872 tests passed and 4 skipped.
- `pnpm exec turbo run test --filter=@bb/host-daemon -- --run
src/machine-auth-proxy.test.ts` — 8/8 tests passed, including WebSocket
upgrade forwarding with machine authentication.
- `bb plugin build plugins/plugin-api-docs` — production build passed.
- `node .github/workflows/check-plugin-sdk-version.mjs` — confirmed the
public surface changed with a version bump.
- Live dev app: installed a throwaway plugin with `auth: "none"`,
connected to `/api/v1/plugins/websocket-e2e/http/v1/echo`, and echoed
`alpha`, `beta`, and binary `[0, 127, 255]` in order.

Tracks BB-93 (layer A).

> AGENT GENERATED
SawyerHood added a commit that referenced this pull request Sep 4, 2026
)

Stack layer 3 (Codex pool track). Prerequisites: #3053 and #3054 below
it. Single-plugin design: Codex joins the existing Account Pool plugin
as a provider adapter.

## Human comments

## What was wrong

Account Pool was coupled to Claude credentials, request rewriting, quota
headers, and refresh behavior, so it could not select or maintain Codex
OAuth accounts. Codex also attempts the Responses WebSocket protocol
before HTTP SSE, while the Codex bridge launched `codex app-server`
without applying provider-contributed per-thread base URL and hub
authentication values. Pointing Codex at the hub therefore required both
a provider-neutral pool core inside the plugin and a session-aware
app-server launch path.

## What changed

- Added an internal provider-adapter contract in `plugins/account-pool`
and preserved the existing Claude behavior behind its adapter.
- Added Codex OAuth import from `~/.codex/auth.json`, serialized
pre-expiry refresh with write-back, ChatGPT account identity storage,
and provider-scoped account selection.
- Added Codex request/header rewriting for ChatGPT's Codex backend and
parsed `x-codex-primary-*` / `x-codex-secondary-*` rate-limit
observations and quota rejections.
- Added authenticated HTTP `POST /v1/responses`, `GET /v1/models`, and
WebSocket `/v1/responses` routes in the existing Account Pool `/http/`
namespace.
- Implemented Codex's downstream envelope session behavior: local
prewarm completion, incremental input expansion after a completed
response, unknown-id failure plus close 1011, upstream HTTPS SSE
forwarding as one JSON event per WebSocket frame, and rotation after
quota rejection.
- Made each WebSocket forward independently abortable. A downstream
close now cancels the hub request signal and upstream fetch/body read,
releases the account's in-flight slot, drops queued frames, and guards
every send against a closing or closed socket.
- Extended the single `bb pool` CLI and Account Pool settings section
with Codex import, provider badges, shared machine tokens, Codex
environment and health contributions, and provider-aware status.
- Made the Codex bridge consume the resolved per-session pool
environment, append in-memory `-c openai_base_url=...` configuration,
add an environment-backed hub header without putting the token in
process arguments, and rebuild a session when pool routing or token
identity changes.
- Updated the configuration reference, generated Guide source, and
builtin bb CLI skill. The original two-plugin extraction plan was
intentionally dropped after the scope decision to keep both providers in
`plugins/account-pool`.

There is no server/host-daemon wire contract change, so
`HOST_DAEMON_PROTOCOL_VERSION` does not need a bump. There is no new
public Plugin SDK surface in this layer.

## Not in this PR

- Keeping `model_provider="openai"` while adding the machine-token
header is not possible with Codex CLI 0.153.2: a live app-server launch
rejects `model_providers.openai` because built-in provider IDs are
reserved and cannot be overridden. The bridge therefore retains the
custom `bb-account-pool` provider ID. That changes the provider recorded
in newly pooled rollouts and may affect Codex's provider-keyed session
grouping, remote compaction, and remote-control behavior. Compatibility
for existing sessions was verified live: a rollout created with
`model_provider: "openai"` completed a direct turn, Account Pool was
enabled, and the same rollout resumed and completed a pooled WebSocket
turn.

## How you verified

The new focused regressions fail on the layer-A base because Codex is
not an Account Pool provider there and the bridge ignores per-session
pool configuration. They now cover import parsing, refresh persistence,
HTTP credential/account-id rewrite, provider-scoped selection, Codex
quota parsing and rotation, WebSocket
authentication/prewarm/incremental/unknown-id/session streaming
behavior, provider env/health, UI import and badges, and token-free
app-server launch arguments.

- `pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool
--filter=bb-plugin-provider-codex --continue --force` — 8/8 tasks
passed; Account Pool 41/41 tests and Codex provider 261/261 tests
passed; both typechecks passed. The added regression closes a downstream
WebSocket during a non-terminating SSE response and proves the upstream
abort signal fires, the account's in-flight count returns to zero, and
no post-close frame is sent.
- `pnpm exec turbo run test typecheck --filter=@bb/server
--filter=@bb/app --filter=@bb/templates --filter=@bb/cli --continue` —
15/15 tasks passed; server 2,209, app 3,872, templates 43, and CLI 530
tests passed; 4 app tests skipped; all selected typechecks passed.
- `bb plugin build plugins/account-pool` — production server and app
bundles built successfully.
- `node scripts/check-provider-literal-ratchet.mjs` — passed with all
core provider literals allowlisted.
- `node .github/workflows/check-plugin-sdk-version.mjs` — passed against
the layer-A SDK surface/version state.
- `git diff --check` — passed.
- Live dev app with Codex CLI 0.153.2 and this machine's imported
account: a pooled WebSocket turn returned `BB93_POOL_WS_OK`, emitted the
Account Pool WebSocket transport marker, and updated five-hour
utilization to 0.4% without fallback/retry noise. The resolved-env card
showed the Account Pool base URL and masked token. After `bb pool
bypass`, a direct turn returned `BB93_DIRECT_OK` while the pool
observation timestamp stayed unchanged. Settings and timeline
screenshots were inspected and attached to BB-93.
- Review E2E with Codex CLI 0.153.2: the attempted built-in `openai`
provider override failed with the reserved-provider validation error.
With Account Pool disabled, rollout
`01a06b6b-a3a4-7de2-9d07-d98af7bc5c90` recorded `model_provider:
"openai"` and returned `BB93_PREPOOL_OPENAI_OK`; after enabling Account
Pool, that same rollout returned `BB93_RESUMED_THROUGH_POOL_OK` over the
plugin WebSocket route and advanced pool utilization from 0.40% to 0.41%
with no account error.

## Review fixes

- Malformed or otherwise failing Codex SSE forwarding now aborts the hub
request and cancels the response reader on every non-EOF exit,
immediately releasing the selected account's in-flight slot.
- Added `releases a Codex request when an upstream SSE event is
malformed`, which sends `data: {\n\n` on a stream that stays open and
proves `response.failed` plus zero in flight through `bb pool status
--json`.
- `pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool
--continue` passes with 42/42 tests and all selected typechecks.

Tracks BB-93 (layer B).

> AGENT GENERATED
ymichael added a commit that referenced this pull request Sep 8, 2026
## Human comments

## What was wrong

[PR #3053](#3053) correctly switched
accounts when the first upstream response declared a model-family limit,
but its short same-account retry used a separate response path. If the
first response was an ordinary short 429 and the retry then declared a
family limit, that second path returned the 429 without trying another
eligible account. [PR #3117](#3117)
later consolidated attempts into one loop and incidentally fixed the
root cause on current `main`, but no regression test covered the
original two-stage response sequence.

## What changed

Added a shipped Account Pooler plugin test using the Plugin SDK
fake-host harness. It drives the exact sequence `sk-one` ordinary short
429, `sk-one` family-limit 429, then `sk-two` success, and asserts the
client receives HTTP 200. There is no production, wire-contract, CLI,
guide, documentation, credential, quota, or unrelated-provider change.

## How you verified

- Red on the [original
landing](c2d5bd3):
HTTP 429 with attempts `sk-one, sk-one`; `sk-two` was not tried.
- Green on current `main`: the focused harness test passed with `sk-one,
sk-one, sk-two` and HTTP 200.
- `pnpm exec turbo run test typecheck build
--filter=bb-plugin-account-pool --concurrency=1 --continue --force` — 6
tasks passed; 10 test files and 262 tests passed; typecheck passed.
- `pnpm exec oxfmt --check plugins/account-pool/src/server.test.ts` and
`git diff --check origin/main...HEAD` passed.

Part of #1552

> AGENT GENERATED
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant